Section: Partnerships and Cooperations

European Initiatives

BigStorage (2015–2018)

Program:

European Training Network (ETN).

Coordinator:

María S. Pérez.

Partners:

Universidad Politécnica de Madrid (UPM), Barcelona Supercomputing Center (PSC), Johannes Gutenberg Universität Mainz, Foundation for Research and Technology - Hellas (FORTH), Xyratex Technology Limited, Deutsches Klimarechenzentrum, CA Technologies, Fujitsu Technology Solutions GmbH, French Atomic Agency CEA, IBM Research Ireland, Bull SAS, and Informatica El Corte Ingles.

Abstract:

The consortium of this Marie-Curie Innovative Training Networks (ITN) BigStorage: Storage-based Convergence between HPC and Cloud to handle Big Data aims at training future data scientists in order to enable them and us to apply holistic and interdisciplinary approaches for taking advantage of a data-overwhelmed world, which requires HPC and Cloud infrastructures with a redefinition of storage architectures underpinning them — focusing on meeting highly ambitious performance and energy usage objectives. KerData mainly collaborates with UPM and PSC 2 co-advised PhD theses).

• Program: EIT ICT Labs.

• Project acronym: EUROPA Activity - Future Cloud Action Line.

• Project title: Big Data Analytics with Apache Flink for Real Business Use-Cases.

• Duration: May 2014–December 2014.

• Coordinator: Gabriel Antoniu, Alexandru Costan.

• Participants: Anirvan Basu, Camelia Ciolac.

• Other partners: TU Berlin (Germany), VTT (Finland), F-Secure (Finland).

• Abstract: In this project, we study the requirements with respect to Big Data analytics today, following several interviews with representative companies from various domains ranging from online mobile gaming to security and logistics. The goal is to identify those requirements that could be addressed by the Apache Flink (formerly known as Stratosphere) platform and apply them in some real-life business scenarios. We first present the state-of-the-art in the field of Big Data analytics, then validate the novel features of Flink. Finally we study how some of the requirements needed by the industry could be addressed by the latter, and illustrate them with 2 real use-cases. To this end, Camelia Ciolac and Anirvan Basu were hired and implemented two demos showing the use of Flink to solve Big Data problems from 2 companies: a mobile games developer (Tribeflame) and a security company (F-Secure), respectively.